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EXAMINER'S ANSWER 



This is in response to the appeal brief filed April 19, 2006 appealing from the Office 
action mailed September 21, 2005. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

6,118,474 Nayar 9-2000 

5,870,135 Glattetal. 2-1999 
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4,908,874 



Gabriel 



3-1990 



6,226,035 



Korein et al. 



5-2001 



(9) Grounds of Rejection 



The following ground(s) of rejection are applicable to the appealed claims: 



Claim Rejections - 35 USC § 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Claims 1-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over US 



[claim 1] 

Nayar teaches the method for generating a selectable perspective view of a 
portion of a hemispherical image scene, comprising the steps of: 

Acquiring an omindirectional image on an image plane using a reflective mirror 
that satisfies a single viewpoint constraint and an image sensor (Col 9 Lines 49-53); 

Defining a perspective viewing window based on configuration parameters (Col 7 
Lines 62-65 and Col 10 Lines 56-65); 



Patent 6,1 18,474 to Nayar in view of US Patent 5,870,135 to Glatt et al. 
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Mapping each pixel in the perspective window with a corresponding pixel value in 
the omnidirectional image on the image plane using the configuration parameters (Col 
10 Line 55-Col 1 1 Line 55 and Col 12 Lines 25-35). 

Nayar does not teach the use of a look-up table based on the configuration 
parameters. 

Glatt teaches the use of a look-up table for storing pixel for displaying a non 
distorted sub image of an original hemispherical image. The look-up table allows for the 
pixels corresponding to those calculated coordinates to be fetched and displayed as if 
the image had been formed by the panning and tilting of a normal camera (Col 8 Lines 
25-55). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine the method of Nayar with the look-up table of Glatt in order to 
display an image as if it had been formed by the panning and tilting of a normal camera 
as taught by Glatt. 
[claim 2] 

Nayar further teaches the method of claim 1 , wherein the configuration 
parameters defined in the defining step include at least one of a zoom distance defined 
as the distance from the focal point of said reflective mirror to said window, a pan angle 
defined as the angle between the x axis and a line through the focal point of said 
reflective mirror perpendicular to the x-y plane and a tilt angle defined as the angle 
between the x-y plane and a vector normal to said window (Col 10 Lines 55-65). 
[claim 3] 
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Nayar further teaches the method of claim 2, wherein the defining step is 
conducted via a user interface through which a user enters data corresponding to at 
least one of a desired zoom distance, pan angle, or tilt angle (Col 7 Lines 62-65). 
[claim 4] 

Nayar further teaches the method of claim 1 , wherein the mapping step includes 
the step of generating a mapping matrix by: 

applying a ray tracing algorithm to each pixel in the perspective viewing window 
to determine a corresponding reflection point on the reflective mirror; and 

projecting each reflection point to a focal point of the image sensor to determine 
the corresponding location in the omnidirectional image on the image plane (Figs. 4, 5 
and 6) 
[claim 5] 

Nayar further teaches the method of claim 4, further comprising the step of 
storing the mapping matrix in a module having a memory. (Fig. 1 A item 125. Computers 
all have memory) 
[claim 6] 

Nayar further teaches the method of claim 1 wherein the step of defining a 
perspective viewing window defines the perspective viewing window as a panoramic 
viewing window. (Col 1 1 Lines 25-30). 

Claims 7-9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nayar and Glatt as applied to claim 1 above, and further in view of US Patent 5,790,181 
to Chahl et al. and US Patent 3,988,533 to Mick et al. 
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[claim 7] 

Nayar and Glatt teach the method of claim 1 as shown above. Nayar and Glatt 
further teaches the use of the system for surveillance purposes (Nayar: Col 1 Lines 25- 
30, Glatt: Abstract) Nayar and Glatt do not teach the use of motion detection. 

Chahl, while he is not specific as to his motion detection means, clearly teaches 
that motion detection can be done in a panoramic surveillance system (Col 6 Lines 25- 
32). However, he does not specify how he does his motion detection. However, at the 
time the invention was made, it was well known in the art of surveillance, that in order to 
detect video motion, the conventional way was to compare subsequent video images 
and if a large enough difference in the images is detected, it is considered motion. As 
proof of the Examiners statements he includes Mick, see abstract. 

Therefore, at the time the invention was made, it would have been 
obvious to one of ordinary skill in the art, to include motion detection capabilities of 
Chahl and Mick in the system of Nayar and Glatt in order to adapt it for use in the 
surveillance application as Nayar says it can be used for. 
[claim 8] 

Nayar further teaches the method of claim 7, comprising the steps of: 
calculating the configuration parameters for the perspective viewing window from 
the anomaly; 

and selectively focusing the perspective viewing window on the anomaly using 
the calculated configuration parameters (Col 10 Lines 32-64). 
[claim 9] 
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Chahl further teaches the use of activating an alarm if at least a portion of the 
residual image exceeds a predetermined threshold (Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to use the alarm 
activating method of Chahl in order to notify security of an anomaly such as an intruder. 

Claims 10-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nayar and Glatt as applied to claim 1 above, and further in view of US Patent 5,686,975 
to Baker, 
[claim 10]. 

Nayar and Glatt teach the method of claim 1 . Nayar suggests the use of the 
method for teleconferencing (Col 1 Line 26). Nayar and Glatt do not teach detecting a 
location of a sound source in the image scene; and adjusting the perspective viewing 
window based on the detected location of the sound source. 

Baker discloses a teleconferencing imaging system that includes a 
panoramic imaging means including an audio detection circuit that can locate the 
source of a sound and image it (Abstract). Therefore, at the time the invention was 
made it was well known in the art, that imaging the source of a sound was desired, thus 
it would have been obvious to one of ordinary skill in the art, to include the audio 
detection system disclosed by Baker into the imaging apparatus of Nayar and Glatt for 
use with teleconferencing, 
[claims 11 and 12] 

At the time the invention was made, use of the internet for transmitting images 
was well known in the art. Once the image is processed as in the case of the current 
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invention, data is data, therefore it doesn't matter that the image came from an omni- 
directional imaging means. Never the less, while Nayar and Glatt are silent on use of 
the Internet, Baker teaches that teleconferencing involves transmitting both the audio 
and video data to a remote site for viewing (Col 1 Lines 16-17). 

Since teleconferencing implies use of the phone system, and access to the 
internet is achieved via the phone network, it would have been obvious to one of 
ordinary skill in the art to transmit the image via a server or any computer capable of 
handling the job in order to provide a suitable teleconferencing system, 
[claim 13] 

The method of claim 1 , further comprising the step of forming a two way 
transmission link between the image sensor and a remote display, wherein the two-way 
transmission link transmits at least one of the omni-directional image, the perspective 
viewing window, and an audio signal. (Teleconferencing is a two way transmission link.) 

Claims 14 and 16-23 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar in view of US Patent 4,908,874 to Gabriel, 
[claim 14] 

Nayar teaches an imaging apparatus for generating a two dimensional image, 
comprising: 

A reflective mirror configured to satisfy an optical single viewpoint constraint for 
reflecting an image scene (Col 9 Lines 49-53). 
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An image sensor responsive to said reflective mirror and that generates two 
dimensional image data signals to obtain an omnidirectional image on an image plane 
(Col 7 Lines 62-65 and Col 10 Lines 56-65); and 

A controller coupled to the image sensor, wherein the controller defines a 
perspective viewing window (Col 9 Line 63-Col 1 0 Line 1 9). Nayar further teaches the 
mapping of each pixel in the perspective window with a corresponding pixel value in the 
omindirectional image on the image plane using the configuration parameters (Col 10 
Line 55-Col 1 1 Line 55 and Col 12 Line 25-35). Nayar further teaches the required 
transformations (zoom, pan) of the hemispherical scene to produce an image from a 
view point (Col 9 Lines 31 -64, Fig 6). Nayar further teaches the use of a computer 
which is well known to contain memory (Fig. 1A item 125). Nayar does not specifically 
teach the memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping 
matrix to perform mapping of pixels from said omindirectional image into said 
perspective viewing window. 

Gabriel teaches the use of matrices to perform transformations such as 
translation, contraction, expansion, rotation and perspective projection (Col 4 Lines 19- 
30, Col 7 Lines 5-64). Gabriel further teaches a complex transformation can be 
produced from the product of simpler ones Col 6 Lines 29-34). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to store the matrices 
of Gabriel in the memory of Nayar in order to perform a transformation without 
processor intensive calculations. 
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[claims 16, 17 r and 20] 

Nayar further teaches the apparatus of claim 14, wherein the mapping step 
includes the step of generating a mapping matrix by: 

applying a ray tracing algorithm to each pixel in the perspective viewing window 
to determine a corresponding reflection point on the reflective mirror; and 

projecting each reflection point to a focal point of the image sensor to determine 
the corresponding location in the omnidirectional image on the image plane (Figs. 4, 5 
and 6) 
[claim 18] 

Nayar further teaches the apparatus of claim 14, wherein the configuration 
parameters defined in the defining step include at least one of a zoom distance defined 
as the distance from the focal point of said reflective mirror to said window, a pan angle 
defined as the angle between the x axis and a line through the focal point of said 
reflective mirror perpendicular to the x-y plane and a tilt angle defined as the angle 
between the x-y plane and a vector normal to said window (Col 10 Lines 55-65). 
[claim 19] 

Nayar further teaches the apparatus of claim 18, wherein the defining step is 
conducted via a user interface through which a user enters data corresponding to at 
least one of a desired zoom distance, pan angle, or tilt angle (Col 7 Lines 62-65). 
[claim 21] 
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Nayar further teaches the apparatus of claim 14, wherein the step of defining a 
perspective viewing window defines the perspective viewing window as a panoramic 
viewing window. (Col 1 1 Lines 25-30). 
[claims 22 and 23] 

Nayar further teaches the apparatus of claim 14, further comprising the step of 
storing the mapping matrix in a module having a memory. (Fig. 1A item 125. Computers 
all have memory) 

Claims 24 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar and Gabriel as applied to claim 14 above, and further in view of US Patent 
5,790,181 to Chahl et al. and US Patent 3,988,533 to Mick et al. 
[claim 24] 

Nayar and Gabriel teach the apparatus of claim 14 as shown above. Nayar and 
Gabriel further teaches the use of the system for surveillance purposes (Nayar: Col 1 
Lines 25-30, Gabriel: Abstract). Nayar and Gabriel do not teach the use of motion 
detection. 

Chahl, while he is not specific as to his motion detection means, clearly teaches 
that motion detection can be done in a panoramic surveillance system (Col 6 Lines 25- 
32). However, he does not specify how he does his motion detection. However, at the 
time the invention was made, it was well known in the art of surveillance, that in order to 
detect video motion, the conventional way was to compare subsequent video images 
and if a large enough difference in the images is detected, it is considered motion. As 
proof of the Examiners statements he includes Mick, see abstract. 
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Therefore, at the time the invention was made, it would have been 
obvious to one of ordinary skill in the art, to include motion detection capabilities of 
Chahl and Mick in the system of Nayar and Gabriel in order to adapt it for use in the 
surveillance application as Nayar says it can be used for. 
[claim 25] 

Chahl further teaches the use of activating an alarm if at least a portion of the 
residual image exceeds a predetermined threshold (Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to use the alarm 
activating method of Chahl in order to notify security of an anomaly such as an intruder. 

Claims 26-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nayar and Gabriel as applied to claim 14 above, and further in view of US Patent 
5,686,975 to Baker, 
[claim 26]. 

Nayar and Gabriel teach the apparatus of claim 14. Nayar suggests the use of 
the method for teleconferencing (Col 1 Line 26). Nayar and Gabriel do not teach 
detecting a location of a sound source in the image scene; and adjusting the 
perspective viewing window based on the detected location of the sound source. 

Baker discloses a teleconferencing imaging system that includes a 
panoramic imaging means including an audio detection circuit that can locate the 
source of a sound and image it (Abstract). Therefore, at the time the invention was 
made it was well known in the art, that imaging the source of a sound was desired, thus 
it would have been obvious to one of ordinary skill in the art, to include the audio 
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detection system disclosed by Baker into the imaging apparatus of Nayar and Gabriel 
for use with teleconferencing, 
[claims 27 and 28] 

At the time the invention was made, use of the internet for transmitting images 
was well known in the art. Once the image is processed as in the case of the current 
invention, data is data, therefore it doesn't matter that the image came from an omni- 
directional imaging means. Never the less, while Nayar and Gabriel are silent on use of 
the Internet, Baker teaches that teleconferencing involves transmitting both the audio 
and video data to a remote site for viewing (Col 1 Lines 16-17). 

Since teleconferencing implies use of the phone system, and access to the 
internet is achieved via the phone network, it would have been obvious to one of 
ordinary skill in the art to transmit the image via a server or any computer capable of 
handling the job in order to provide a suitable teleconferencing system, 
[claim 29] 

The imaging apparatus of claim 31, further comprising: a remote display coupled 
to the image sensor; a first speaker and first microphone coupled to the image sensor; 
and a second speaker and second microphone coupled to the remote display, wherein 
the first and second speakers and first and second microphone form a two-way 
transmission link between the image sensor and the remote display. (Teleconferencing 
is a two way transmission link.) 

Claims 30-38 and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar and Gabriel in view of US Patent 6,226,035 to Korein. 
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[claim 30] 

Nayar and Gabriel teach the apparatus of claim 14. While it is unclear as to what 
shape Nayar's reflector is, i.e. the equation that defines it is not that of a parabola nor a 
sphere, it does resemble that of a hyperbola and it satisfies the single viewpoint 
limitation see Nayar column 9 lines 49-53, never the less at the time the invention was 
made, as admitted by the applicant in his own specification, it was well known in the art 
that a hyperbolic could be used to view a panoramic scene from a single viewpoint, and 
has already been in use for such an application, see Korein column 9 line 59 wherein he 
teaches the use of a hyperbolic mirror. 

Therefore at the time the invention was made, it would have been obvious to one 
of ordinary skill in the art to use a hyperbolic mirror in the invention of Nayar and Gabriel 
since the single viewpoint constraint is satisfied using a hyperbolic mirror. 
[Claims 31, 38 and 44]. 

Nayar teaches an imaging apparatus for generating a two dimensional image, 
comprising: 

A reflective mirror configured to satisfy an optical single viewpoint constraint for 
reflecting an image scene (Col 9 Lines 49-53). 

An image sensor responsive to said reflective mirror and that generates two 
dimensional image data signals to obtain an omnidirectional image on an image plane 
(Col 7 Lines 62-65 and Col 10 Lines 56-65); and 

A controller coupled to the image sensor, wherein the controller defines a 
perspective viewing window (Col 9 Line 63-Col 1 0 Line 1 9). Nayar further teaches the 
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mapping of each pixel in the perspective window with a corresponding pixel value in the 
omindirectional image on the image plane using the configuration parameters (Col 10 
Line 55-Col 1 1 Line 55 and Col 12 Line 25-35). Nayar further teaches the required 
transformations (zoom, pan) of the hemispherical scene to produce an image from a 
view point (Col 9 Lines 31-64, Fig 6). Nayar further teaches the use of a computer 
which is well known to contain memory (Fig. 1A item 125). Nayar does not specifically 
teach the memory for storing a mapping matrix for each of a plurality of sets of said 
configuration parameters in a parameter space, said controller using a said mapping 
matrix to perform mapping of pixels from said omindirectional image into said 
perspective viewing window. 

Gabriel teaches the use of matrices to perform transformations such as 
translation, contraction, expansion, rotation and perspective projection (Col 4 Lines 19- 
30, Col 7 Lines 5-64). Gabriel further teaches a complex transformation can be 
produced from the product of simpler ones Col 6 Lines 29-34). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to store the matrices 
of Gabriel in the memory of Nayar in order to perform a transformation without 
processor intensive calculations. 

While it is unclear as to what shape Nayar's reflector is, i.e. the equation that 
defines it is not that of a parabola nor a sphere, it does resemble that of a hyperbola 
and it satisfies the single viewpoint limitation see Nayar column 9 lines 49-53, never the 
less at the time the invention was made, as admitted by the applicant in his own 
specification, it was well known in the art that a hyperbolic could be used to view a 
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panoramic scene from a single viewpoint, and has already been in use for such an 
application, see Korein column 9 line 59 wherein he teaches the use of a hyperbolic 
mirror. 

Therefore at the time the invention was made, it would have been obvious to one 
of ordinary skill in the art to use a hyperbolic mirror in the invention of Nayar and Gabriel 
since the single viewpoint constraint is satisfied using a hyperbolic mirror, 
[claim 32] 

Nayar further teaches the imaging apparatus of claim 31 , wherein the reflective 
mirror creates a one-to-one correspondence between pixels in the omnidiredional image 
and pixels in the perspective viewing window. (Col 10 Lines 20-31 .) 
[claim 33] 

Nayar further teaches the imaging apparatus of claim 31 , wherein the controller 
maps the omnidirectional image to the perspective viewing window by mapping each 
pixel in the perspective viewing window with a corresponding pixel value in the 
omnidirectional image. (Col 12 Lines 25-34) 
[claim 34] 

Nayar further teaches the imaging apparatus of claim 14, wherein parameters 
defining the perspective viewing window include at least one of a zoom distance defined 
as the distance from the focal point of said reflective mirror to said window, a pan angle 
defined as the angle between the x axis and a line through the focal point of said 
reflective mirror perpendicular to the x-y plane and a tilt angle defined as the angle 
between the x-y plane and a vector normal to the perspective viewing window. (The 
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Examiner is assuming that this claim was supposed to be dependent from claim 31 not 
14. In either case see (See Nayar column 10 lines 55-65). 
[claim 35] 

Nayar further teaches the imaging apparatus of claim 34, further comprising a 
user interface through which a user enters data corresponding to at least one of a 
desired zoom distance, pan angle, or tilt angle. (Col 7 lines 62-65) 
[claim 36] 

Nayar further teaches the imaging apparatus of claim 31 , wherein the controller 
generates a mapping matrix by applying a ray tracing algorithm to each pixel in the 
perspective viewing window to determine a corresponding reflection point on the 
reflective mirror and then projecting each reflection point to a focal point of the image 
sensor to determine the corresponding location on the omni directional image. (Figs. 4, 
5 and 6) 
[claim 37] 

Nayar further teaches the imaging apparatus of claim 31 , wherein the 
perspective viewing window is a panoramic viewing window. (Col 1 1 lines 25-30) 

Claims 39 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nayar, Gabriel and Korein as applied to claim 31 above, and further in view of US 
Patent 5,790,181 to Chahl et al. and US Patent 3,988,533 to Mick et al. 
[claim 39] 

Nayar, Gabriel and Korein teach the apparatus of claim 31 as shown above. 
Nayar and Gabriel further teaches the use of the system for surveillance purposes 
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(Nayar: Col 1 Lines 25-30, Gabriel: Abstract). Nayar and Gabriel do not teach the use of 
motion detection. 

Chahl, while he is not specific as to his motion detection means, clearly teaches 
that motion detection can be done in a panoramic surveillance system (Col 6 Lines 25- 
32). However, he does not specify how he does his motion detection. However, at the 
time the invention was made, it was well known in the art of surveillance, that in order to 
detect video motion, the conventional way was to compare subsequent video images 
and if a large enough difference in the images is detected, it is considered motion. As 
proof of the Examiners statements he includes Mick, see abstract. 

Therefore, at the time the invention was made, it would have been 
obvious to one of ordinary skill in the art, to include motion detection capabilities of 
Chahl and Mick in the system of Nayar, Gabriel, and Korein in order to adapt it for use 
in the surveillance application as Nayar says it can be used for. 
[claim 40] 

Chahl further teaches the use of activating an alarm if at least a portion of the 
residual image exceeds a predetermined threshold (Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to use the alarm 
activating method of Chahl in order to notify security of an anomaly such as an intruder. 

Claims 41-43 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nayar, Gabriel and Korein as applied to claim 31 above, and further in view of US 
Patent 5,686,975 to Baker, 
[claim 41]. 
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Nayar, Gabriel, and Korein teach the apparatus of claim 31 . Nayar suggests the 
use of the method for teleconferencing (Col 1 Line 26). Nayar and Gabriel do not teach 
detecting a location of a sound source in the image scene; and adjusting the 
perspective viewing window based on the detected location of the sound source. 

Baker discloses a teleconferencing imaging system that includes a 
panoramic imaging means including an audio detection circuit that can locate the 
source of a sound and image it (Abstract). Therefore, at the time the invention was 
made it was well known in the art, that imaging the source of a sound was desired, thus 
it would have been obvious to one of ordinary skill in the art, to include the audio 
detection system disclosed by Baker into the imaging apparatus of Nayar, Gabriel and 
Korein for use with teleconferencing, 
[claim 42] 

At the time the invention was made, use of the internet for transmitting images 
was well known in the art. Once the image is processed as in the case of the current 
invention, data is data, therefore it doesn't matter that the image came from an omni- 
directional imaging means. Never the less, while Nayar, Gabriel and Korein are silent 
on the use of the Internet, Baker teaches that teleconferencing involves transmitting 
both the audio and video data to a remote site for viewing (Col 1 Lines 16-17). 

Since teleconferencing implies use of the phone system, and access to the 
internet is achieved via the phone network, it would have been obvious to one of 
ordinary skill in the art to transmit the image via a server or any computer capable of 
handling the job in order to provide a suitable teleconferencing system. 
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[claim 43] 

The imaging apparatus of claim 31, further comprising: a remote display coupled 
to the image sensor; a first speaker and first microphone coupled to the image sensor; 
and a second speaker and second microphone coupled to the remote display, wherein 
the first and second speakers and first and second microphone form a two-way 
transmission link between the image sensor and the remote display. (Teleconferencing 
is a two way transmission link.) 

(10) Response to Argument 

In response to the Appellant's arguments related to claims 1-6, it is noted by the 

Examiner that this argument was originally responded to in the Final Rejection mailed 

on September 21 , 2005 as cited below: 

In regards to the applicants arguments related to the rejection of claims 1- 
6, 14 and 16-23, the applicant argues that Glatt does not teach the "mapping of 
pixels from an omnidirectional image into a perspective window as claimed. 
Rather, Glatt is merely teaching selecting or fetching specific pixels, without any 
mapping, distortion correction or other processing, to simulate panning and 
tilting." As shown in the previous Office Action, Nayar teaches the mapping from 
the omnidirection (which is hemispherical) to cartesian coordinates using 
calculations (Col 1 1 Lines 7-9). Nayar is lacking the use of look-up tables for the 
mapping. Glatt teaches the mapping from a fish-eye lens (which is 
hemispherical) to catesian coordinates using a look-up table (Col 8 Lines 28-43). 
As taught by Glatt, this mapping is in order to present an image as if it had been 
formed by the panning and tilting of a normal camera (Col 8 Lines 40-43). As 
omindirection and fish-eye lens are both hemispherical it would have been 
obvious to adapt the look-up table of Glatt to be used by Nayar in order to display 
an image as if it had been formed by the panning and tilting of a normal camera 
as taught by Glatt. 

In response to applicant's argument that there is no suggestion to combine 
the references, the examiner recognizes that obviousness can only be 
established by combining or modifying the teachings of the prior art to produce 
the claimed invention where there is some teaching, suggestion, or motivation to 
do so found either in the references themselves or in the knowledge generally 
available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 
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USPQ2d 1596 (Fed. Cir. 1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 
(Fed. Cir. 1992). In this case, as shown above, both Nayar and Glatt teach the 
conversion of pixels from a hemispherical image to a normal camera image. 
Therefore it would have been obvious to one of ordinary skill to use method of 
look-up tables as taught by Glatt in order to simplify the mapping in Nayar. 
In regards to the applicant's specific argument related to claim 2, the look-up 
tables of Glatt are used only to present that it was well known in the art to use 
look-up tables in order to map pixels from a hemispherical image to a normal 
camera image. By using this known technique the calculations used in Nayar 
could be simplified by the use of look-up tables. Therefore the look-up tables 
would be based on the configuration parameters of Nayar. 

To further clarify the Examiner's position, Glatt is used only to teach the use of a look-up 

table in order to simplify the transformation of a hemispherical image to a normal 

camera image. 

The Appellant argues that Glatt does not teach or suggest how mapping would 
be performed using a look-up table for an image that comes, not from a fisheye lens, 
but from a reflective mirror. Appellant further argues, the mapping function of Glatt 
cannot be imported into the system taught by Nayar because Nayar does not operate 
on an image produced with a fish-eye lens. It is not the intension of the Examiner to 
suggest that the look-up table of Glatt can be imported into the system of Nayar. It is 
the intension of the Examiner to use a look-up table (as taught by Glatt) with the system 
of Nayar in order to provide a means to translate the hemispherical image into a normal 
camera image without continuously having to perform complex calculations. 

Specifically, Glatt teaches populating the look-up table (LUT) with the pre- 
calculated coordinates for the fisheye image (Col 8 Lines 34-37). These coordinates 
are calculated using complex equations (equations 9 and 10 of Col 8). Nayar teaches 
the use of continuously calculating the coordinates as depicted in the program of 
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Appendix I. It would have been obvious to one of ordinary skill in the art to pre-calculate 
the coordinates of Nayar and store them in a look-up table as taught by Glatt in order to 
prevent the need to continuously perform complex calculations when converting a 
hemispherical image into a normal camera image. 

In regards to arguments related to claims 2, 3, 18, 19, 34 and 35, the Appellant 
states that Nayar and Glatt fail to teach or suggest the claimed configuration parameters 
including a zoom distance, pan angle, and tilt angle as claimed. It is noted by the 
examiner that the claims only require at least one of the configuration parameters. It is 
noted in the Final Rejection that Nayar teaches the use of a zoom feature. This zoom is 
further defined by Nayar using Figure 6 (Col 8 Lines 31 -41 ). Glatt further teaches the 
use of zoom distance, pan angle and tilt angle parameters (Col 7 Lines 40-46 and Col 8 
Lines 4-15). These parameters are well known in the art for use with converting 
hemispherical images to normal camera images as shown by Nayar and Glatt. 

In regards to the arguments related to claims 4, 5, 20 and 36, Nayar teaches the 
mapping steps as shown in Figures 4-6 (Col 9 Line 63-Col 10 Line 30). 

In regards to the arguments related to claims 14, 16-23, 31-38 and 44, the 
Appellant states that Gabriel does not teach a mapping matrix for each of a plurality of 
sets of configuration parameters. This requirement is very broad and is interpreted by 
the examiner to be a mapping matrix for each configuration parameter. Therefore, each 
set contains one configuration parameter. Gabriel teaches the importance of four 
special cases of transformation matrices (translation, scaling, rotation and shearing) 
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which can be combine to perform any desired transformation (Col 7 Lines 5-64). Thus, 

by storing these four matrices any transformation can be performed. 

The Appellant further argues the use of matrices of Gabriel with a method using 

omidirectional image acquired with a "reflective mirror 7 '. This argument was responded 

to in the Final Office Action mailed September 21 , 2005 as noted below: 

In regards to the applicants arguments related to claim 14, the applicant 
argues "Gabriel does not teach or suggest anything regarding omnidirectional 
images and, consequently, cannot teach the claimed memory with mapping 
matrix for mapping pixels from an omnidirectional image to a perspective viewing 
window." Gabriel teaches the use of simple matrices used to perform complex 
warping of images such as to a fish-eye image (Col 4 Lines 24-67). The ability of 
Gabriel to convert from a normal image to a hemispherical image would 
inherently mean that the use of the matrices may be used to convert from 
hemispherical image to a normal image, as such in operation would be the 
inverse of the warping. Again, as with Glatt, Gabriel is used to show a known 
method in the art for converting pixel coordinates. Therefore it would have been 
obvious to one of ordinary skill in the art to use the simple matrices of Gabriel in 
order to produce the complex transformations of Nayar without processor 
intensive calculations. 



Further, it has been shown above that both fish-eye images and omindirectional images 
are both hemispherical and therefore it would have been obvious to one of ordinary skill 
in the art at the time of the invention to use the teachings for converting one 
hemispherical type image to a normal image for converting another type of 
hemispherical image to a normal image. 

In regards to claim 38, the Appellant argues that Nayar, Gabriel and Korein fail to 
teach or suggest memory containing a predetermined mapping matrix for every set of 
configuration parameters. Assuming every set contains one configuration parameter, it 
has been shown above that Gabriel teaches the use of a matrix for each of the four 
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configuration parameters needed to perform any type of transformation (Col 7 Lines 5- 
64). 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 
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For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 
Erick Rekstad 
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